6.2 Web Services

  1. Motivations
    • Examples of service servers - Google Maps service; PayPal service; ...
    • Overall diagram
    • How to use your Google Account to sign in to other sites or apps? 1-1-m model? 1-m model?

  2. Learning objectives
    • List the three components in Web Services.
    • Explain how RESTful web services are different.
    • List the two types of APIs.
    • List the four REST operations with their corresponding HTTP methods.
    • List the naming conventions used in REST APIs.
    • Understand "Same-Origin Policy (SOP)" and "Cross-Origin Resource Sharing (CORS)"
    • Problem solving: Implementation of a RESTful service that provides the access to MongoDB
      • Define the REST API between the client and the server.
      • Define the library API that is used for the client.
      • Implement a web app with the libray, and a server-side program.
      • Use "Express" middleware.

  3. A problem to solve in this class - MongoDB web service
    • Let's develop a simple MongoDB web service.
    • What kind of services?
      • A user can access a MongDB database stored on a server computer system from a web application. The user can create/insert a document in a collection. The user can read/find documents, update documents and delete documents.
    • Requirements?
      • Requests use REST API, and responses use JSON strings.
      • User management: Open a connection with username and password.
      • The connection to MongoDB should be maintained only for a certain time, e.g., 1 minute.
      • The connection needs to be refreshed at every transactions.
      • Supported operations with a collection: open and close, insert, find, update, delete

  4. What are web services
    • Read several paragraphs in Web Services.
      • How do you define web services?
        Web app components that provide certian information through open protocols
      • What language is used to describe web services?
      • What protocols are used to access web services?
      • List the two types of uses of web services.
    • Here is another good reference - tutorialpoint - Web Services.
      • Web services are open standard based web applications that interact with other web applications for the purpose of exchanging data.
      • Questions
        • Which protocol(s) is(are) used?
        • Where can you find web service descriptions?
        • How to use web services?
        • Is automation possible?

  5. Representational State Transfer (REST)ful web services

  6. Example - Use your Google Account to sign in to other sites or apps

  7. Example - Google Maps API Web Services
    • An example of the 1-m (a client - multiple servers) model, where the web service is used by the client-side.
    • How to use Google Maps web services, not how to develop the web services
    • Read all in Google Maps JavaScript API - Overview.
      • This tutorial includes the API, how to use Google Maps web service, from the perspective of client-side programming. The API between the client and the server is not explained.
      • Here is an example.
      • Here is the source code. 
        <!DOCTYPE html>
<html>
  <head>
    <style type="text/css">
      html, body, #map-canvas { height: 100%; margin: 0; padding: 0;}
    </style>
    <script 
      src="https://maps.googleapis.com/maps/api/js?language=en?key=YOUR_API_KEY">  <!-- Assigned to my Google account -->
                                                                                   <!-- Look closely. Path? -->
    </script>
  </head>
  <body>
    <div id="map-canvas"></div>
    <script type="text/javascript">
      function initialize() {
        var mapOptions = {
          center: { lat: 50.671441, lng: -120.36273},  // TRU
          zoom: 14
        };
        var map = new google.maps.Map(document.getElementById('map-canvas'), mapOptions);  // Where is google defined?
      }
      google.maps.event.addDomListener(window, 'load', initialize);  // Where is google.maps defined?
    </script>
  </body>
</html>
      • Let's try to understand the above code.
        • Information hiding - Use of a JavaScript library for the client-side app, in which the API between the client and the server is implemented
        • API key for Google services
        • Creation of a related object
      • Google Maps API requires API keys. You can obtain the API keys - Get the API key.
    • Here is another excellent tutorial - Google Maps Tutorial.
    • There are many very interesting features.
    • Does Google Maps service use any complex protocol or description?
    • Here is the full API reference - Google Maps Javascript API V3 Reference.

  8. How to develop a web service - what do we need to consider?
    • The services that will be provied by the web service server
    • The REST API between the client and the server
    • Implementations:
      • Client-side:
        • A libray that hides all the detail client-side implementation of REST API
        • The API how to use the above libray
      • Server-side:
        • Support REST API
        • Support CORS
      • Cross-Origin Resource Sharing (CORS) - A request for a resource (like an image or a font) outside of the origin is known as a cross-origin request. 'Outside of the origin' includes not only different server computer systems but also differet port numbers. Web browsers generally blocks the traffic coming from outside of the origin for security. The idea of CORS is to use HTTP header options to allow cross-origin resource requests. Read CORS Tutorial: A Guide to Cross-Origin Resource Sharing for further understanding.
        • Example when SOP can be useful:

  9. REST APIs
    • APIs in between the client and the server
    • Let's watch
    • Let's read 10+ Best Practices for Naming API Endpoints.
    • Uniform interface
      • Request:
        • CRUD (Create, Read, Update, Delete) resource operations
          Create/open/newHTTP POSTSupported in HTML5
          Read/get/retrieveHTTP GETSupported in HTML5
          Update/changeHTTP PUTNot supported in HTML5How to send the HTTP PUT data?
          Delete/closeHTTP DELETENot supported in HTML5How to send the HTTP DELETE data?
        • Naming conventions for URIs that are used to identify resources (Note resources don't have to be files or data in a DB.)
          • Nouns - E.g., /tokens/{id}, not /getToken, where {...} just means a parameter
            In an actual URL, .../tokens/12, not {12}.
          • Pluralized resources - E.g., /tokens or /tokens/{id}/value for a singleton resouce
          • Forward slashes for hierachy - E.g., /tokens, /tokens/{id}, /tokens/{id}/value
          • Comma for lists - E.g., /tokens/{id1},{id2}
          • Query with ? - E.g., /tokens?username=David
          • Lowercase letters and dashes - E.g., /tokens?username=David-Williams, not /Tokens?Username=David_Williams
          • No trailing forward slash - E.g., Not /tokens/
          • Meaningful names, no shortened names, not file extension, ... For example, users, memos, collections, channels, friends, regions, ...
        • The combination of an HTTP method and nouns in an URI gives a meaningful serive request. (Note that the combination is called a route in Express.js.)
      • Response: JSON/XML

  10. MongoDB web service - a class project
    • Let's develop a simple DB service.
    • What kind of services?
      • A user can access a MongDB database stored on the server from a web application. The user can insert a document in a collection. The user can find documents, update documents and delete documents.
      • To make the project simple, let's use one database for all service users.
    • Requirements?
      • Requests use REST API, and responses use JSON strings.
      • Open a connection with username and password.
      • Connection should be maintained only for a certain time, e.g., 1 minute.
      • The connection needs to be refreshed at every transactions.
      • Operations with a collection: open and close, insert, find, update, delete
    • Let's try this implementation, MongoDB Service. It uses an Express app server with port number 8088, and the Express app server is used to access MongoDB. (Note that you need to use http, not https, to run this example.)
    • What we need to implement is

    • REST API
      • Uniform interface - See 6.2.9
        • Request:
          • CRUD (Create, Read, Update, Delete) resource operations
            Create/open/newHTTP POSTSupported in HTML5
            Read/get/retrieveHTTP GETSupported in HTML5
            Update/changeHTTP PUTNot supported in HTML5How to send the HTTP PUT data?
            Delete/closeHTTP DELETENot supported in HTML5How to send the HTTP DELETE data?
          • Naming conventions for URIs that are used to locate resources
            • Nouns - E.g., /tokens/{id}, not /getToken
            • Pluralized resources - E.g., /tokens or /tokens/{id}/value for a singleton resouce
            • Forward slashes for hierachy - E.g., /tokens, /tokens/{id}, /tokens/{id}/value
            • Comma for lists - E.g., /tokens/{id1},{id2}
            • Query with ? - E.g., /tokens?username=David
            • Lowercase letters and dashes - E.g., /tokens?username=David-Williams, not /Tokens?Username=David_Williams
            • No trailing forward slash - E.g., Not /tokens/
            • Meaningful names, no shortened names, not file extension, ...
          • The combination of an HTTP method and nouns in an URI gives a meaningful serive request.
        • Response: JSON

      • Idea:
        • User management
        • How to use?
          1. Open a connection with usernamd and password
          2. Open a collection
          3. CRUD operations
          4. Close the connection

      • API:
        • Create/Register a new user
          • Request - POST /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Delete a user
          • Request - ??? /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'

        • Create/Open a new connection (i.e., get a new token)
          • Request - ??? /tokens?username=...&password=...
          • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
        • Delete/Close the connection
          • Request - ??? /tokens/{tokenid}
          • Response - '{"result":"true"|"false", "explanation":"..."}'

        • Create/Open a new collection or an existing collection
          • Request - ??? /collections?tokenid=...&name=...
          • Response - '{"collectionid":"...", "explanaton":"..."}', where if collectionid is negative, then error
        • Create/Insert a new document
          • Request - ??? /collections/{collection_id}?tokenid=...&document=..., where the document value should be a JSON string.
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Read/Find documents
          • Request - ??? /collections/{collection_id}?tokenid=...&query=..., where the query value should be a JSON string for MongoDB query.
          • Response - '{"result":"true"|"false", "documents":"...", "explanation":"..."}', where the documents value is a JSON string of an array of found documents.
        • Update documents
          • Request - ??? /???/{???}?tokenid=...&query=...&document=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Delete documents
          • Request - ??? /???/{???}?tokenid=...&query=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'

    • Cient-side library API
      • var db = new TRUMongoDBService(host) - Connect to the server
        • This object will hold
          • host
          • connection token id
          • collection name and id
        • How to use?
          • If a user were not registered, then the user should be registered first.
          • .open()
          • .collection()
          • CRUD operations
          • .close()
      • db.register(username, password, callback(result)) - Register a new user
      • db.unsubscribe(username, password, callback(result)) - Delete a user
      • db.open(username, password, callback(result)) - Sign in; Connection
      • db.collection(collection_name, callback(result)) - Select a collection
      • Insert, find, update, delete: Operations can be strings of MongoDB operations.
        db.insert(document, callback(result)),
        db.find(query, callback(result)),
        db.update(query, document, callback(result))
        db.delete(query, callback(result)),
      • db.close() - Close the connection
      • Note that token_id and collection_id are hidden and not included in the above API

    • In a REST server, for the processing of REST API, let's study a Node framework, Express - Fast, unopinionated, minimalist web framework for Node.js, "with a myriad of HTTP utility methods and middleware at your disposal, creating a robust API is quick and easy."
      • Basic routing: Routing refers to determining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method (POST, GET, PUT, DELETE, and so on). Each route can have one or more handler functions, which are executed when the route is matched. Each route, a combination of path in URL and a HTTP method, can be handled using the following structure: 
        app.METHOD(PATH, HANDLER);  // METHOD: a HTTP request method; PATH: path in URL

      • How to respond to different routes?
        Example of "Hello World!": 
        const express = require('express');  // "Express" framework
const app = express();
const port = ???;  // Use your port number.

// Allow Cross-domain requests, i.e., CORS (Cross-Origin Resource Sharing)
//   Why do we need this?
app.all('/*', function(req, res, next) {  // all(): any HTTP method; /*: any routes
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers", "X-Requested-With");
    res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");  // Default: only GET and POST
    next();  // Express middleware function; To continue to next operations
});

// A route with GET method and '/'
app.get('/', (req, res) => {
  res.send('GET: Hello World!')  // send(): combination of write() and end()
})

// A route with POST method and '/'
app.post('/', (req, res) => {
  res.write('POST: Hello World!')
  res.end()
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
      • How to run the above example?
      • Trial 1. Let's copy the above example in test.js, not test.sjs, and run test.js with your port number, and test it. The example URLs to test can be http://198.162.21.138:yourport/, http://198.162.21.132:yourport/tests, and http://198.162.21.132:yourport/tokens/1
        Another idea to test test.js is to use the next code. Probably, the Chrome web browser will block the use of http instead of https in this trial, because the current web page uses https. You may need to redisplay the current web page with http://198.162.21.132:8080/~mlee/comp4620/Winter2025/6.%20web_services/2.%20web_services.html, and test test.js.



      • How to get query and parameters from a request?
        POST, PUT, DELETEGET
        express.json();
        req.body.queryname        		req.query.queryname
        req.params.idreq.params.id

      • How to get query from a request?
        E.g., .../users?username=john&password=topsecret. Let's use two middlewares, express.urlencoded() and express.json(), for POST, PUT, and DELETE queries. Here is an example. 
        const express = require('express')
const app = express()

app.use(express.urlencoded({extended:false}));
app.use(express.json());

const port = ???  // Use your port number.

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

// For CORS
...

// A route with GET method and '/'
app.get('/', (req, res) => {
  res.send('GET: Hello World!')  // send(): combination of write() and end()
})

// A route with POST method and '/'
app.post('/', (req, res) => {
  res.write('POST: Hello World!')
  res.end()
})

// A route with GET method and '/users/'
app.get('/users', (req, res) => {
  res.send('GET: username=' + req.???.username + "; password=" + ????)
})

// A route with POST method and '/users'
app.post('/users', (req, res) => {
  res.send('POST: username=' + req.???.username + "; password=" + ????)
})
      • Trial 2. Let's run the above example code in test.js using your port number, and test test.js. The example URLs to test can be http://198.162.21.132:.../users?username=john&password=topsecret



      • How to get parameters?
        E.g., .../tokens/1 and .../tokens/1,2. Let's use req.params.id to get a parameter or parameters. Here is an example. 
        const express = require('express')
const app = express()
const port = ???  // Use your port number.

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

...

// GET method with '/tokens/{id}'
app.get('/tokens/:id', (req, res) => {  // Be careful with ':'
  res.send("GET: /tokens; id=" + req.???.id);
})

// POST method with '/tokens/{id}'
app.post('/tokens/:id', (req, res) => {
  res.send("POST: /tokens; id=" + req.???.id);
})
      • Trial 3. Let's add the above code in test.js used in Trial 2 and run test.js using your port number, and let's test it. The example URLs to test can be http://198.162.21.132:.../tokens/1 and .../tokens/1,2.


      • Trial 3.5. Let's add the very similar code in test.js, that is used to delete/close tokens, and run test.js using your port number, and let's test it. The example URLs to test can be http://198.162.21.132:.../tokens/1 and .../tokens/1,2.



      • Read all in Basic routing.

    • Implementation of the server side code - Library to access MongoDB, and REST server
      • Requirements:
        • A new user can sign up.
        • An existing user needs to connect to the system, i.e., SignIn.
        • One collection for each user for the simplicity
        • In the beginning, the collection name is given from the user. The name of collection and username are not submitted for the next CRUD operations. Instead, a token id is submitted.

      • Reserved collections in MongoDB:
        • Users - triples of unique username, password, and collection name that is currently being used
          {username:..., password:..., collectionname:...}, ...
        • Tokens - count for token ids (i.e., id for a connection), and pairs of username and token id
          {token:...},
          {username:..., tokenid:...}, ...
        • How to find the name of current working collection with a token id?
          Token id => username => collection name

      • Model, i.e., library, to access MongoDB
        • model.js
        • API 
          ready(callback)
    true or false
close()

usernameExists(username, callback)
    true or false
validateUsernamePassword(username, password, callback)
    true or false
registerUser(username, password, callback)
    true or false
deleteUser(username, password)
    nothing to return

getNewToken(username, callback)  // Create/Open a connection
    (true, token id), or
    (false, -1)  // error case
deleteToken(token_id, callback)  // Delete/Close a connection
    true or false
    
collection(token_id, collection_name, callback)  // use collection_name for token_id
    (true, 1), or 
    (false, -1) or (false, -2)
insertOne(token_id, doc, callback)  // insert one document
    true or false
find(token_id, query, callback)  // find documents
    (true, result) or  // result: array of documents
    false
updateMany(token_id, query, newdoc, callback)  // update documents
    true or false
deleteMany(token_id, query, callback)  // delete documents
    true or false
        • Example of SignUp (Create/Register a user):
          ready() => registerUser()
        • Example of Unsubscribe (Delete a user):
          ready() => deleteUser()
        • Example of SignIn/Connection (Open a connection, and ...):
          ready() => validateUsernamePassword() => getNewToken()
        • Example of Close (Close a connection):
          ready() => deleteToken() => close()

      • Preparation: 
        .../current_working_directory/
    - node_modules          You may need to install 'mongodb@4.17.2' and 'express'.
    - model.js              You need to download this file.
    - rest_server.js        You need to write this program.

      • Here is an example REST sever, rest_server.js, that uses the above library. 
        const express = require('express');
const model = require('./model.js');

const app = express();

const server = app.listen(???, function () {  // User your port number
    let host = server.address().address;
    let port = server.address().port;
    console.log("Listening at http://%s:%s", host, port)
})

// for POST, PUT, and DELETE query
app.use(express.urlencoded({extended:false}));
app.use(express.json());

// To support CORS
app.all('/*', function(req, res, next) {  // /*: any routes
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers", "X-Requested-With");
    res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");  // Default: only GET and POST
    next();  // Express middleware function; To continue to next operations
});

// Route operations

app.get('/', function (req, res) {  // HTTP GET method
    res.send("Welcome to TRU MongoDB Web Service!");
})

// SignUp (Create/Register a user)
//   Request - POST /users?username=...&password=...
//   Response - '{"result":"true"|"false", "explanation":"..."}'
app.post('/users', function (req, res) {  // HTTP POST method
    var username = req.body.username;
    var password = req.body.password;
    console.log("POST /users: username = %s, password = %s", username, password);
    model.ready(function(result) {
        if (result) {
            model.registerUser(username, password, function(result) {
                if (result)
                    res.send(JSON.stringify({result:true, explanation:""}));
                else
                    res.send(JSON.stringify({result:false, explanation:"Username exists"}));
            });
        }
        else
            res.send(JSON.stringify({result:false, explanation:"Connection error"}));
    });
});

...

      • Trial 4. Let's implement rest_server.js that supports SignUp and Unsubscribe, and test it. Here are the related REST API messages that are submitted to the server.
        • REST requests:
          • Create/Register a new user - SignUp
            • Request - ??? /users?username=...&password=...
            • Response - '{"result":"true"|"false", "explanation":"..."}'
          • Delete a user - Unsubscribe
            • Request - ??? /users?username=...&password=...
            • Response - '{"result":"true"|"false", "explanation":"..."}'
        • API for model.js: 
          ready(callback)
    true or false
registerUser(username, password, callback)
    true or false
deleteUser(username, password)
    nothing to return
        • Example of SignUp (Create/Register a new user): ready() => registerUser()
        • Example of Unsubscribe (Delete a user): ready() => deleteUser()



      • Trial 5. Let's add the code in rest_server.js that supports SignIn and Close, and test it. Here are the related REST API messages that are submitted to the server.
        • REST requests:
          • Create/Open a new connection - SignIn
            • Request - ??? /tokens?username=...&password=...
            • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
          • Delete/Close the connection - Close
            • Request - ??? /???/{tokenid}
            • Response - '{"result":"true"|"false", "explanation":"..."}'
        • API for model.js: 
          ready(callback)
    true or false
close()
validateUsernamePassword(username, password, callback)
    true or false
getNewToken(username, callback)  // Create/Open a connection
    (true, token id), or
    (false, -1)  // error case
deleteToken(token_id, callback)  // Delete/Close a connection
    true or false
        • Example of SignIn/Connection: ready() => validateUsernamePassword() => getNewToken()
          Note that the callback function in getNewToken() needs to use two parameters.
        • Example of Close: ready() => deleteToken() => close()



      • You can also test rest_server.js with Demo for TRU MongoDB Web Service with your port server number.

      • You can implement the next REST requests in rest_server.js to complete TRU MongoDB Web Service.
      • Create/Open a new collection or an existing collection
        • Request - ??? /collections?tokenid=...&name=...
        • Response - '{"collectionid":"...", "explanaton":"..."}', where if collectionid is negative, then error
      • Create/Insert a new document
        • Request - ??? /collections/{collection_id}?tokenid=...&document=..., where the document value should be a JSON string.
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Read/Find documents
        • Request - ??? /collections/{collection_id}?tokenid=...&query=..., where the query value should be a JSON string for MongoDB query.
        • Response - '{"result":"true"|"false", "documents":"...", "explanation":"..."}', where the documents value is a JSON string of an array of found documents.
      • Update documents
        • Request - ??? /???/{???}?tokenid=...&query=...&document=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Delete documents
        • Request - ??? /???/{???}?tokenid=...&query=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'

    • Implementation of the client side library - Library to use REST API
      • API
        • var db = new TRUMongoDBWebService(host) - Connect to the server
          • This object will hold
            • host
            • connection token id
            • collection name
            • collection id
          • How to use
            • If a user were not registered, then the user should be registered first.
            • .open()
            • .collection()
            • CRUD operations
            • .close()
        • db.register(host, username, password, function(result) {...}) - Register a new user
        • db.unsubscribe(host, username, password, function(result) {...}) - Delete a user
        • db.open(host, username, password, function(result) {...}) - Sign in
        • db.collection(collection_name, function(result) {...}) - Select a collection
        • Insert, find, update, delete: Operations can be strings of MongoDB operations.
          db.insert(document, function(result) {...}), // document and query are JSON strings of objects
          db.find(query, function(result) {...}), // result is an object
          db.update(query, document, function(result) {...})
          db.delete(query, function(result) {...}),
        • db.close() - Close the connection
        • Note that token_id and collection_id are not included in the above API
      • TRUMongoDBWebService() 
        function TRUMongoDBWebService() {
    this.host = "";  // with open()
    this.connection_token_id = -1;
    this.collection_name = "";
    this.collection_id = -1;
    
    this.register = function(host, username, password, callback) {
        ...
    }
    this.unsubscribe = function(host, username, password, callback) {
        ...
    }
    ...
}
      • register() 
            this.register = function(host, u, p, callback) {
        $.ajax({
            url: host + "/users",  // POST /users?username=...&password=...
            method: "post",
            data: { username: u, password: p },
            success: function(data) {
                data = JSON.parse(data);
                callback(data);
            }
        });
    }
      • ...
      • Trial 6. Let's implement the above client-side library that supports SignUp and Unsubscribe, and test it. Here are the related messages that submitted to the server. (If you did not complete Trial 4 and 5, you may use tr6_server.js.)
        • SignUp: Create/Register a new user - register()
          • Request - POST /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Unsubscribe: Delete a user - unsubscribe()
          • Request - DELETE /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'


      • Trial 7. Let's implement the above client-side library that supports SignIn and Close, and test it. Here are the related messages that submitted to the server. (If you did not complete Trial 4 and 5, you may use tr6_server.js.)
        • SignIn: Create/Open a new connection - open()
          • Request - POST /tokens?username=...&password=...
          • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
        • Close: Delete/Close the connection - close()
          • Request - DELETE /tokens/{tokenid}
          • Response - '{"result":"true"|"false", "explanation":"..."}'



    • Implementation of the client side app

  11. References for further information